1. Field of the Invention
The present invention relates to an image segmentation method of segmenting a particular portion out of an image necessary for simulating a design, and to an apparatus therefor.
2. Description of the Prior Art
Systems using abundant expressions of images have been vigorously studied and developed, such as presentation systems in shops. In order to more effectively utilize image information, image simulations such as synthesis of background, change of object colors, and the like have been extensively employed for simulation of designs of household electric appliances.
In order to carry out image simulation, it is necessary to segment a particular object out of an image. This is called an image segmentation technology. For instance, separation of only the portion of a car from an image including the car and taken on a street is an image segmentation technology, and insertion of this portion in a scenery of the country is an image simulation.
The image segmentation technology can be roughly divided into two. The first method uses color information, as is described in, for example, "Full Color Image Processor SHAIP and Its Applications", O plus E, January, 1989, pp. 131-142. When, for example, a red car is to be separated from the background according to this method, it is judged whether the color of each pixel in the image is in a certain range of red color, and it is discriminated, depending upon the result of this judgement, whether it is an object to be segmented. Concretely speaking, when a pixel of the i-th row and j-th column has a color of R, G and B components r(i, j), g(i, j), b(i, j), and when these components have minimum and maximum threshold values .SIGMA.rmin, .SIGMA.rmax, .SIGMA.gmin, .SIGMA.gmax, .SIGMA.bmin, and .SIGMA.bmax, the following range is segmented, i.e., EQU .SIGMA.rmin&lt;r(i, ])&lt;.SIGMA.rmax EQU and EQU .SIGMA.gmin&lt;g(i, j)&lt;.SIGMA.gmax EQU and EQU .SIGMA.bmin&lt;b(i, j)&lt;.SIGMA.bmax
This is the most general segmentation method which uses color information and has been employed by many companies as represented by the SHAIP mentioned above.
The discrimination processing is to judge whether or not conditions for discrimination (condition (2) in this case) are satisfied by the pixels in some region or of the whole image. The region (mask) consisting of a set of pixels that are judged to satisfy the above conditions should, most desirably, be in agreement with the region that is to be separated from other regions in the image, but are not necessarily in agreement therewith due to the ability of the discrimination processing.
In order to decide Rmin, etc. in this processing, histograms are often prepared for each of red, green and blue colors and are displayed.
The histograms show the distributions of values of red, green and blue colors of pixels in the whole image or in some regions, and show the values related to red, green and blue components and information related to the numbers of pixels having such values. In the above-mentioned example of a red car, a red mailbox that happens to exist in the background is also segmented. Therefore, the processing range must be limited to a real space (i-j space).
There is a special technology, chromakey method, in the technologies. The chromakey method has a feature that an object to be segmented such as a person is disposed in front of a background board of a predetermined color (such as green), and an image is photographed. When the portions other than the above-mentioned predetermined color is the objects to be segmented, the object is desirably segmented.
The second method uses edge information of the image, i.e., uses information of portions where values (color, brightness, etc.) of pixels in the image change. These methods have been studied in order to comprehend and recognize scenes in an image, and are described in "Computer Vision", Prentice-Hall, Inc., Chapter 4 Boundary Detection, K. H. Ballard, C. B. Brown, 1982. Basically, portions having large differences in the color or density among the neighboring pixels in an image are found as edges and are then connected to find its contour. The steps of finding edges are described below.
In the case of a monochromatic density image, the i, j image is expressed as a(i, j). The i-direction represents a lateral direction and the j-direction represents a longitudinal direction.
(1) Differentiation of first order.
Lateral direction: a(i+1, j)-a(i, j), PA1 Longitudinal direction: a(i, j+1)-a(i, j), ##EQU1## etc. (2) Differentiation of second order. PA1 Lateral direction: a(i+1, j)-2a(i, j) +(a(i-1, j), PA1 Longitudinal direction: a(i, j+1)-2a(i, j)+a(i, j-1), PA1 Both directions: a(i+1, j)+a(i-1, j)+a(i, j +1)+a(i, j-1)-4a(i, j) PA1 etc.
(3) Zero crossing.
A point at which the differentiation of second order changes from positive to negative or from negative to positive. When information of intensity is used instead of that of position, the degree of change (corresponds to the differentiation of third order) is used.
The above two methods deal chiefly with a full-color image having 16.7 million colors per pixel and a monochromatic density image having 256 tones per image. Other various processings can be effected for simple images such as a binary image having two tones per pixel. For example, when a black object exists on a white background without overlapping other image patterns, boundaries of an object to be segmented are specified, and the specified boundaries are traced in order to segment the object.
Furthermore, there is another method of directly inputting coordinates of the contour of an object using an external input device such as a mouse instead of segmenting the image by the aforementioned image processing technology. That is, the cursor on the screen is located in synchronism with the operation and position of the mouse, and the cursor is caused to run along the contour of the object to be segmented, thereby to use the locus thereof as the contour.
As a recent study concerning the second method, furthermore, a system called SNAKE which uses a dynamic model is described in the Proceedings of First International Conference on Computer Vision, 1987, IEEE Computer Society Press, pp. 259-268.
Among the aforementioned conventional technologies, however, the first method that uses color information is effective when the distribution of object regions to be segmented is isolated, has a rectangular parallelopiped shape, and can be favorably discriminated. However, it is difficult to adapt a single reference of discrimination to the whole image where the color tone changes delicately depending upon the regions.
Even when separate histograms are used for red, green and blue colors, furthermore, the color distribution in the image is not fully grasped, and threshold values for discrimination are not properly set, making it difficult to favorably segment the image.
Among the aforementioned conventional technologies, furthermore, the first method which uses color information cannot be adapted except in the case where the object to be segmented has a uniform color and a color distribution which is different from other regions.
According to the second method which uses edge information among the aforementioned conventional technologies, a number of edges appear because of noises in the image, in addition to the edges of the object. Moreover, edges often disappear even in positions where a contour of the object should be formed. Therefore, it is difficult to selectively connect only the edges of the object. Many of these technologies hold only in limited environments such as in lines of factories, or require a tremendously long period of time.
Furthermore, it is difficult to apply the method of the binary image mentioned in the third place to images except for the binary images and multi-value images having small numbers of tones.
The last-mentioned method of directly specifying contour positions through an external input device has a problem that laborious work is required for correctly specifying the contour positions.